Papers with parameter removal
OptiPrune: Effective Pruning Approach for Every Target Sparsity (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for model pruning only perform optimally within specific sparsity ranges. |
| Approach: | They propose a pruning method that reduces model size by eliminating redundant parameters . they compare it with OptiPrune, which adapts non-uniform sparsity with adaptive deviation . |
| Outcome: | The proposed method reduces model size and maintains performance despite large size and high computational demands. |
DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs (2025.emnlp-main)
Copied to clipboard
Minxuan Lv, Zhenpeng Su, Leiyu Pan, Yizhe Xiong, Zijia Lin, Hui Chen, Wei Zhou, Jungong Han, Guiguang Ding, Wenwu Ou, Di Zhang, Kun Gai, Songlin Hu
| Challenge: | Existing sparsification methods like pruning can lose model knowledge through parameter removal. |
| Approach: | They propose a novel approach that achieves sparsification by partitioning pre-trained FFN layers into computational blocks. |
| Outcome: | The proposed approach achieves superior performance across language modeling and downstream tasks under equivalent computational constraints. |